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This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional 
Patent Application Serial No. 60/420,700 filed October 23, 2002 entitled: METHOD AND 
SYSTEM FOR SUPERCOMPRESSION OF COMPRESSED DIGITAL VIDEO, which is 
incorporated herein by reference. This application also claims priority under 35 U.S.C. § 119(e) 
to U S Provisional Patent Application Serial No. 60/420,504 filed October 23, 2002, entitled 
METHOD AND SYSTEM FOR USING ARITHMETIC CODING IN SUPERCOMPRESSION 
OF COMPRESSED DIGITAL VIDEO which is incorporated herein by reference. 



FIELD OF THE INVENTION 

The present invention relates generally to digital data transmission, and more 
specifically to digital data compression. Even more specifically, the present invention relates to 
15 accommodation of multiple digital data compression formats. 

BACKGROUND OF THE INVENTION 

As is known, a pixel is a dot of light displayed on a video display device with a 
20 certain color. The term "frame" has been employed to refer to a matrix of pixels at a given 
resolution. For example, a frame may comprise a 640 by 480 rectangle of pixels containing 480 
rows having 640 pixels each. In an uncompressed state, the amount of data required to represent 
a frame is equal to the number of pixels times the number of bits associated with each pixel to 
represent color. Thus, in a pure black and white image lacking any grayscale shades, a pixel 
25 could be represented by one bit where "1" represents white and "0" represents black. More 
typically, in modem full-color displays a single pixel is represented by 8-bits, 16-bits or 32-bits. 
THUS, a single uncompressed 32-bit frame at a resolution of 640 by 480 would require (32 * 640 
*480) 9.8 million bits, or 1.2 Megabytes of data. 

Digital video is the display of a series of frames in sequence (e.g., a motion 
30 picture is composed of 24 frames displayed every second). Thus, one second of uncompressed 
32 bit frames at a resolution of 640 by 480 requires (1.2 * 24) 29.5 Megabytes of data. 
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Digital video compression is a complex process that may use any of a variety of 
techniques to transform ("encode") a unit of uncompressed video data into a unit of data that 
requires fewer bits to represent the content of the original uncompressed video data. The 
resultant encoded data is capable of being transformed using a reverse process ("decode") that 
provides a digital video unit of data that is either identical to the original data ("lossless 
compression") or visually similar to the original data to a greater or lesser degree ("lossy 
compression"). 

Modem techniques of digital video compression can achieve very high levels of 
compression with relatively low loss of visual quality. As a general rule, modem techniques of 
digital video compression are very computationally intensive and the degree of compression 
varies directly with the amount of computationally intensity. Anything that adds to 
computational intensity over and above the decoding techniques is undesirable. In particular, m 
virtually all fomis of modem compression the amount of data in each compressed video frame 
will vary, sometimes to a great extent. This maximizes compression, but at the cost of makmg 
15 the processing power needed to decode the frames inconsistent. 

Typical digital video encoders have been used to reduce the size of a stream of 
uncompressed digital video data. FIG. 1 is a block diagram of a conventional digital video 
encoder 1 25, which is comprised of a video processmg unit 1 10 and an entropy compression umt 
115 Digital video encoder 125 uses motion estimation and motion compensation to exploit 
temporal redundancy in some of the uncompressed video frames 120 that comprise its mput 
signal in order to generate compressed video output. 

During operation of video encoder 125, video processing unit 110 accepts 
uncompressed video frames 120 and appUes one or more video and signal processing techniques 
to such frames. THese techniques may include, for example, motion compensation, filtenng, 
two-dimensional ("2D") fransfomiation, block mode decisions, motion estimation, and 
quantization. TTie associated event matrices include some or all of: a skipped blocks binary 
matrix, a motion compensation mode (e.g. intra/forwardA,i-directional) matrix, a motion 
compensation block size and mode matrix (e.g. 16x16 or 8x8 or interlaced), a motion vectors 
matrix, and a matrix of transformed and quantized block coefficients. 
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In tt.e special case of a lossy video encoder, these technique, aim to retain image 
information that is important to toe human eye. The video processing unit 110 produces data 
streams 124 that are more suitable than .he uncompressed video ftames 120 as an mput ^ 
entropy coding algoritlum. Conventionally. ^ intennediate data streams 124.. would 

5 comprise transform coefficients with dear statistical redundancies and motion vectors. As an 
example, video processing unit 110 can apply a block DCT or other transfonn function to the 
output of motion compensation and quantize the resulting coefficients. 

An entropy coding technique such as Hufflnan Coding can then be apphed by 
entropy compression unit 115 to the data streams 124.. in order to p^duc. a compressed stream 

,0 130 Theentropycompressionnni.ll5eomp.essestheda.as.reamswithnolossofinformauon 

by exploiting their statistical redundancies. The compressed s^ 130 output by entropy 
compr^sion unit US is of significantly smaller size than both the uncompressed video frames 
120 and the intermediate data stream 124 infonnation. 

Shnilarly. as shown in FIG. 2. a conventional digital video decoder 230 may be 
,5 divided into two logical components: entropy decompression unit 235 and video processing umt 
240 Entropy decompression unit 235 receives the compressed data stream 103 and outputs data 
streams 250.. typically comprising motion vectors and transform (or quantized) coefHctent. 
Video processing unit 240 takes the data stream output 250.. from decompression um. 235 and 
perfonns operations such as mofion compensation, inverse quantization, and mverse 2-D 
20 transfomiation in order to reconstruct the uncompressed video frames. 

MPEG (Motion Pictures Experts Group) and the ISO (Intemational Standards 
Organization) have produced intemational standanis specifying such video compression and 
decompression algorithms of the type implemented by the encoder 125 and decoder 230 
^.ively. These s^mdards mdude MPEG-1. MPEG-2, MPEG-4, H.261. H.263, and pernut 
25 equipment, hardware, and software fiom different manufacU^ers to exchange compressed v.d» 
with ease in accord»ce with the applicable algorithm. The MPEG^ video c^np^sston 
technique is very efficient, and is generally considered to produce virtually "incompresstble 

output. . 

Since standardization of these algorithms, research has revealed motion 

30 compensation, transform, and entropy coding techniques that can compress video wrth 
equivalent subjective quality as the old« techmques while producing sigmficantly less 
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compressed data. Work is currently underway at MPEG and ISO to produce a new standard 
"H.26L" that will incorporate some of the newer algorithms. 

There are a number of problems that arise when a superior video compression 

technology or standard becomes available: 

legacy encoders will not be able to compress video according to the new 

standards; 

legacy decoders will not be able to decompress video according to the new 
standards; 

legacy compressed video content stored on disk or tape needs to be 
recompressed according to the new standard in order to take advantage of 
the newer techniques; and 

video that is recompressed will have been subject to two lossy processes 
and will thus be of an inferior quality. 

What is needed, then, is a method and system for compressing and decompressing 
video using new standards as they become available, without subjecting the video to two lossy 
processes. 
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SUMMARY OF THE INVENTION 

In one embodiment, the invention can be characterized as a method, and a 
processor readable medium containing processor executable instructions for carrying out the 
m«hod, for converting digital video from a first compressed forma, to a second compressed 
fbnnat, the method comprising: receiving an input distal video stream in said first compressed 
format; demultiplexing said input digital video stream so as to generate a multtphcty of 
constituent data streams, wherein said constituent data streams include a compressed data 
stream; decompressing said compressed data str«un so as to generate a decompressed data 
stream; compressing said de^mpressed data stream so as to generate a recompressed data 
stream, wherein said recompressed data stream is more compres^ ^ said compr^ data 
stream and wherein said recompressed data stream conveys identical semantic infonnation as 
said compressed data stream; and multiplexing said recompressed data stream and a subset of 
said constituent data streams that was not subject to said decompressing into an output digital 
video stream in said second compressed format. 

m another embodiment, the invention can be charactedzed as a mediod, and a 
processor readable medium contaming processor executable insti^ctions for carrying out ttie 
method, for converting digital video ftom a first compressed fonnat to a second compressed 
fomtat. the method comprising: receiving an itiput digital video stieam in said first compressed 
tomtat; demultiplexing said input digital video stieam so as to generate a multipUcty of 
eonstititent data streams, wherein said constituem data streams include a compressed data 
stteam; decompressing said compressed data stieam so as to gene^te a decompressed data 
stream; compressing said decompressed data stieam so as to generate a recompressed data 
stream, wherein said .compressed data stream conveys identical semantic information as satd 
compressed data stieam; and multiplexing said recompressed data stieam wifl. a subset of sard 
constituent data streams that was not subject to said decompresdng into an output digtud vtdeo 

Stream in said second compressed format. 

In a further embodiment, the invention may be characterized as a method, and a 
, processor readable medium containing processor executable instructions for carrying out the 
method, for transforming uncompressed video frames into at least two compressed formats, the 
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method comprising: receiving uncompressed video frames; processing said uncompressed video 
frames into intermediate data streams; applying a first entropy compression format to at least 
some of said intermediate data streams so as to generate a first set of compressed data streams; 
applying a second entropy compression format to at least some of said intermediate data streams 
5 so as to generate a second set of compressed data streams; multiplexing at least said first set of 
compressed data streams so as to generate a video stream in accordance with said first format; 
and multiplexing at least said second set of compressed data streams so as to generate a video 
stream in accordance with said second format 

In yet another embodiment, the invention may be characterized as a method for 
10 converting digital video from a first compressed fonnat to a second compressed format, the 
method comprising: receiving an input digital video stream in said first compressed format; 
demultiplexing said input digital video stream so as to generate one or more compressed data 
streams and an uncompressed data stream; decompressing one of said one or more compressed 
data streams so as to generate a decompressed data stream; compressing said decompressed data 
15 stream so as to generate a recompressed data stream; compressing said uncompressed data 
stream so as to generate a newly compressed data stream; and multiplexing said recompressed 
data stream and said newly compressed data stream into an output digital video stream in said 

second compressed format. 

In yet another embodiment, the invention may be characterized as a method for 
20 converting digital video from a first compressed format to a second compressed format, the 
method comprising: receiving an input digital video stream in said first compressed format; 
demultiplexing said input digital video stream so as to generate a plurality of compressed data 
streams; decompressing one of said pluraUty of compressed data streams so as to generate a 
decompressed data stream; compressing said decompressed data stream so as to generate a 
25 recompressed data stream, wherein said recompressed data stream is more compressed than said 
one of said plurality of compressed data streams; and multiplexing said recompressed data 
sfream with another of said plurality of compressed data streams into an output digital video 
stream in said second compressed format. 
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BMEF DESCMPTION OF THE DRAWINGS 



FIG. 1 is a block diagram depicting a conventional digital video encoder 

5 according to the prior art; 

FIG. 2 is a block diagram depicting a conventional digital video decoder 

according to the prior art; 

FIG. 3 is a block diagram depicting a compressing converter according to an 

embodiment of the present invention; 
10 FIG. 4 is a block diagram depicting one embodiment of the compressing 

converter of FIG. 3 ; 

FIG. 5 is a flow chart depicting steps carried out by the compressing converter of 

FIG. 4 according to one embodiment; 

FIG. 6 is a block diagram depicting a dual output digital video encoder according 

15 to an embodiment of the present invention; 

FIG. 7 is a block diagram depicting a dual output digital video encoder according 

to another embodiment of the present invention; 

FIG. 8 is a block diagram depicting a dual output digital video encoder according 

to yet another embodiment of the present invention; 
20 FIG. 9 is a block diagram depicting one embodiment of the video processors of 

FIGS. 6, 7 and 8; 

FIG. 10 is a block diagram depicting a dual input digital video decoder accordmg 

to an embodiment of the present invention; 

FIG. 1 1 depicts four different bitmaps for determining a context value according 

25 to an exemplary embodiment; 

FIG. 12 is a flow chart showing steps carried out during supercompression of 

digital video event matrices using arithmetic coding; 

FIG. 13 is a flow chart showing steps carried out during decompression of 
supercompressed digital video event matrices using arithmetic coding; 
30 FIG. 14 depicts an exemplary event matrix to be supercompressed; and 

FIG. 1 5 depicts a partially populated event matrix to be fiirther decompressed. 
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DETAILED DESCRIPTION 

As is described herein, the present invention relates to a system and method for 
furtier compression of compressed video content, which is also referred herein as 

5 supercompression of compressed video content. In accordance with one aspect of the mvenfon, 
a compn^ssing converter receives a digital video signal in a first compressed format, ,.e„ 
fom«t A The compressmg converter re-encodes some or all of the elements of the compressed 
stteam using differ«>t entropy coding techniques than those used to generate the original stream. 
This re-encoding may be done in such a way that none of the signal information pertammg to the 

,0 video in the original stream is lost or modified. The re-encodmg gen«ates an output digital 

video signal compliant with a second format, i.e., format B. 

The output digital video signal may then be processed by a decompressmg 
converter, h. operation, the decompressing converter receives a digital video signal in the format 
B and generates an output digital video signal that complies with format A. In an exemplary 
,5 embodim».t, format B i. designed or chosen so that it uses identical video processmg steps as 
fom«t A but may be entropy-encod^d to a higher compression than format A, Tims the process 
of converting ftom format A to format B results in a compressed bitstream that is sigmficantly 
smaller. 

Advantageously, the invention according to several embodiments allows users of 
20 a legacy video format to take advantage of new entropy compression techniques whilst retaming 
compliance with their existing encoders, decoders and compressed content. 

The teachings of the invention may also be utilized within a dual output encoder 
and a dual input decoder. As is described below, these devices may be used in applications 
where an encoder or decoder together with a compression or decompression converter would 

25 otherwise be needed. 

Turning now to FIG. 3, a compressing converter 305 according to one 

embodiment of the present invention transforms one type of digital video data stream to a 

different, compressed representation of the exact same information. As shown m FIG. 3, 

compressing converter 305 includes an entropy decompression unit 310 configured to process 

30 compressed video frames 312 of video compression format A The compressing converter 305 

further includes an entropy compression unit 315 designed to compress intermediate data streams 
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3208-c produced by decompression unit 310 into compressed video frames 330 of video 

compression format B. 

As discussed herein, the compressed video frames 312 of video compression 
format A include a multiplicity constituent data streams, which the entropy decompression unit 
5 310 processes to provide the multiplicity of intermediate streams 320a-c. In an exemplary 
embodiment, entropy decompression unit 310 decompresses some of the compressed format A 
constituent data streams while passing through other compressed format A constituent data 
streams so as to generate intermediate data streams 320aK: comprising both uncompressed data 
streams and data streams compressed according to format A entropy compression techniques. 
10 The entropy compression unit 315 then recompresses one or more of the uncompressed data 
streams using entropy compression algorithms of format B so as to generate recompressed data 
streams. The recompressed data streams are then multiplexed with the compressed data streams 
so as to generate the compressed video frames 330 of video compression format B. 

In other embodiments, the entropy compression unit 310 may decompress all of 
15 the constituent streams of the format A compressed stream 312 so that all of the decompressed 
data streams may be recompressed with enfropy compression techniques that provide greater 
compression with respect to format A compression techniques. One of ordinary skill in the art 
will appreciate that there may be a tradeoff between the amount of increased compression gained 
by decompressing and recompressing (according to format B) all of the constituent data streams 
20 of the format A compressed stream 312 and the added time necessary to decompress and 
recompress all of the constituent data streams. As a consequence, some of the format A 
constituent data sfreams may be passed through the format A entropy decompression unit 310 
and the format B entropy compression unit 330 without being decompressed and recompressed, 
if decompressing and recompressing them does not provide an amount of compression gain 
25 commensurate with the time associated with the recompression process. 

hi an exemplary embodiment, the entropy compression algorithm of format B is a 
lossless, yet more highly compressive algorithm than the format A algorithm so that the resulting 
compressed video frames 330 of video compression format B provides a more compressed 
representation of the exact same information as is contained in the format A compressed video 
30 frames 312. One exemplary compression algorithm, which may be implemented as the format B 
compression algorithm, is described herein with reference to FIGS. 11-15. It should be 
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recognized, however, that other compression techniques may be used to supercompress 
compressed video content without departing from the scope of the present invention. 

In an exemplary embodiment, the format A compressed stream 312 is formatted 
in accordance with the ISO MPEG-4 video standard, which makes extensive use of Huf6nan 
5 coding. In this embodiment the format B compression scheme uses arithmetic coding for 
syntactic elements: block coded/not coded patterns, block coding intra/inter modes, motion 
compensation block mode selection, block sizes, and DCT or other transform coefficients, for a 
subset of the video frames. Format B in this embodiment may share the original MPEG-4 
entropy coding for the remaining data steam elements. This embodiment is also suitable for 
10 other DCT-based compressed video formats. 

In another embodiment, the format A compressed stream 312 is in accordance 
with the H.264 Context-based Adaptive Variable Length Coding (CAVLC) standard and the 
format B compressed video stream 330 is in accordance with the H.264 Context Adaptive Binary 

Arithmetic Coding (CABAC). 

As one of ordinary skill in the art will appreciate, the compressing converter 305 
may be adapted so that it receives an arithmetically coded format B video stream and generates a 
format A output stream in accordance with ISO MPEG-4. 

Referring next to FIG. 4, shown is a block diagram of one embodiment of the 
compressing converter of FIG. 3. As shown, the compressing converter 305 includes an entropy 
decompression unit 410 coupled with an entropy compression unit 415. As shown, the entropy 
decompression unit 410 and the entropy compression unit 415 are specific embodiments of the 
entropy decompression unit 310 and the entropy compression unit 315 described with reference 
to FIG. 3. The entropy decompression unit 410 is configured to receive the format A 
compressed video data stream 312 and generate intermediate data streams 420a-<., which are 
25 provided to an entropy compression unit 415. The entropy compression unit 415 then generates 
the format B compressed digital video data stream 330 from the intermediate data streams 420a.<j. 
While referring to FIG. 4 simultaneous reference will be made to FIG. 5, which is a flow chart 
depicting steps of an exemplary embodiment, which are carried out by the compression converter 
305 when converting a compressed video data stream of format A to a compressed video data 
30 stream of format B. 
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As shown in FIG. 4, the compressed video data stream 312 of video compression 
formatAisinitiallyreceivedbyaformatAdemultiplexer within the entropy d^^^^^^ 
410 (Step 502). The fomiat A demultiplexer then demultiplexes the compressed video data 
stream 312 into its constituent data streams 431.^ (Step 504). In the present embodiment, the 
5 constituent data streams include a plurality of compressed constituent streams 431. 431. and 
431d and at least one uncompressed data stream 431e. It should be recognized that each of the 
constituent streams 431.^ illustrated in FIG. 4 represent a different processing path that may be 
taken by constituent streams of the format A compressed signal 3 1 2. Moreover, each constituent 
stream 43U-. may include one. two or multiple syntactic data elements of the format A 

10 compressed signal 305. 

For example, in embodiments where format A is MPEG-4 video according to 
ISO/IEC 14496-2 specification, a first constituent data stream 431a may include "motion vector^' 
and '«block» planes; a second constituent data stream 431. may, but not necessarily must, include 
"mcbpc » "cbpy" and "block" planes; a third constituent stream 431c includes "acpred," "mcsel" 

15 and "nJt coded" planes; and a forth-constituent stream 431a, which is neither decompressed or 
recompressed, may include any of the above-mentioned planes depending upon whether it is 
advantageous to send a particular stream through the compression converter 305 without either 

decompressing or compressing the stream. 

As shown, the first of the compressed constituent streams 431a is a prediction- 
20 coded stream (e.g., a motion vector stream), which is decompressed by a first decodmg module 
432 to produce a decompressed prediction-coded stream 435 (Step 506). The decompressed 
prediction-coded stream 435 is then provided to the data prediction module 436, which m 
cooperation with the stored predictors 438. decodes the prediction-coded stream 435 so as to 
generate a first intermediate stream 420a (Step 508). The first intermediate stream 420a is then 
25 received by a prediction encoding module 442. which in cooperation with stor«i predictors 440. 
prediction encodes the first intermediate stream 42O3 according to format B to produce an 
encoded stream 443 (Step 510). The encoded stream 443 is received by a first variable length 
encoding module 444, which compresses the encoded stream according to format B entropy 
compression techniques so as to generate a compressed prediction coded stream 449a (Step 512). 



30 



11 



As shown in FIG. 4, a second constituent data stream 431b is decompressed by a 
second decoding module 434 to produce a second intermediate data stream 420b (Step 514). The 
second intermediate data stream 420b is then received and compressed by a second variable 
length encoding module 446 according to format B entropy compression techniques so as to 
5 generate a recompressed data stream 449b (Step 516). 

As shown, uncompressed constituent stream 431c is passed through the entropy 
compression unit 410 to the entropy compression unit 415 as an uncompressed intermediate 
stream 402c (Step 518). This stream is a stream of data that is not compressed according to 
format A, but is passed along to the entropy compression unit 415 where it is compressed 
10 according to format B entropy compression techniques so as to generate a newly compressed 

data stream 449c (Step 520). 

As shown in FIG. 4, another compressed constituent stream 431d is passed 
through the entropy decompression unit 410 as a compressed intermediate stream 420<„ which is 
received by the format B multiplexer 450 (Step 522). The format B multiplexer, as depicted in 
15 FIG. 4, receives and multiplexes the compressed prediction stream 449. the recompressed data 
stream' 449b, the newly compressed data stream 449c and the compressed intermediate stream 
420d into the compressed digital video data stream 330 of format B (Step 524). 

Advantageously, in the embodiment described with reference to FIG. 4, the 
compression converter 305 is configured to implement format B compression techniques that 
20 utilize the same or different prediction encoding/decoding as format A. Specifically, the 
prediction decoder 436 may be configured to remove the prediction encoding regardless of its 
format to provide an intermediate stream 420, that is encoded by the prediction encoder 442 

according to format B. 

It should be recognized, however, that in some embodiments, the format B 
25 compression uses the same prediction coding as format A. In these embodunents, the prediction 
decoder 436 and the prediction encoder 442 are umxecessary and need not be incorporated into 
the compression converter 305. Likewise, Steps 508 and 510 need not be carried out, and the 
decompressed prediction-coded stream 435 may be provided directly to the variable length 
encoder 444 for compression according to format B entropy compression techniques. 

30 
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Referring now to FIG. 6. a block diagram is provided of a dual output encoder 
605 operative to generate compressed video output in either a format A, a format B, or in both 
formats simultaneously. As shown, dual output encoder 605 includes a video processmg umt 
610 configured to receive uncompressed video data 608 and generate intermediate data streams 
5 630.. which are received by both a first entropy compression unit 615 operative in accordance 
With format A and a second entropy compression unit 620 configured to produce compressed 
output consistent with format B. Again, the format B compression utilizes compression 
techniques (e.g., arithmetic coding) that provide increased compression relative to format A 
compression techniques (e.g., Huffinan coding). In one embodiment, format B provides such 

1 0 increased compression without losing data. 

In the embodiment of FIG. 6, the two entropy compression units 615, 620 are 
configured to process the same syntactic elements provided by the video processing unit 610. As 
a consequence, the dual output encoder 605 only requires a single video processmg unit 610. 
Thus the dual output encoder 605 of the present embodiment requires fewer resources (e.g. 

15 syst«nmemory,programsize,siUconarea,electricalpower)thanwouldberequiredifasepm^^ 

video processing unit were implemented for each compression unit 615, 620. 

In an exemplary embodiment, the format B entropy compression unit 620 
compresses one or more of the intermediate streams 630a-c, which the format A entropy 
compression unit 615 does not compress. In these embodiments, the compression gains provided 
20 by the format B entropy compression unit 620 include gains due to improved compression 
techniques (e.g., arithmetic compression techniques) and gains due to compressing streams, 

which are not compressed at all. 

M one embodiment for example, the video processing unit 610 processes the 
uncompressed video stream 608 according to the ISO/IEC 144496-2 specification to produce 
25 intermediate streams 630.., which include a "not_coded" syntactic element. Tliis element is 
compressed by the format B entropy compression unit 620, but is not compressed by the format 

A compression unit 615. 

Refemng next to FIG. 7, shown is a block diagram depicting a dual output 
encoder 700 according to an alternative embodiment of the present invention. As shown, the 
30 dual output encoder 700 includes a first video processing unit 710, which receives an 
uncompressed data stream 708 and provides intermediate data streams 730 to a format A entropy 
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compression unit 715, which generates a format A compressed stream 718 by compressmg one 
or more of the intemiediate data streams 730 according to format A compression techniques. 
The dual output encoder 700 also includes a second video processing unit 712 which receives an 
uncompressed data stream 708 and provides intermediate data streams 740 to a format B entropy 
compression unit 720, which generates a fomiat B compressed stream 722 by compressing one 
or more of the intermediate data streams 740 according to format B compression techniques. 

In an exemplary embodunent, the format B compression unit 720 uses improved 
compression techniques (e.g., arithmetic coding) relative to those used by the format A 
compression unit 715 (e.g., Huffinan coding) to generate the format B compressed stream 722 

without a loss of image data. 

In several embodiments, the first and second video processing units 710, 712 are 
configured to generate identical intermediate streams 730, 740, which are compressed according 
to different compression techniques. M some of these embodiments, however, the format B 
entropy compression unit 720 compresses some syntactic elements of the intermediate data 
streams 730, 740, which the format A compression unit 715 does not compress. 

Referring next to FIG. 8, shown is a block diagram of yet another embodiment of 
a dual output encoder 800. As shown, the uncompressed stream 708 is converted into a fomiat A 
compressed stream 718 by the video processing unit 710 and the fomiat A entropy compression 
unit 715 in the same mamier as described with reference to FIG. 7. M this embodiment, 
however, the fomiat A compressed stream 718 is received by the compressing converter 305 
which generates a format B compressed stream 802 as described with reference to FIG. 3. 

Referring next to FIG. 9, shown is a block diagram depicting one embodiment of 
a video processing unit 900 capable of implementing the video processing unit 610 of FIG. 6 and 
the video processing units 710, 712 of FIG. 7. As shown, a motion compensation module 904 
within the video processing unit 900 receives an uncompressed video stream 902 and processes 
each frame within that stream. Each frame is passed to the motion estimation unit 906 together 
with zero or more reference frames that were previously stored by the motion compensation unit 
904. The motion estimation unit 906 performs a searching algorithm to discover good motion 
vectors and mode decisions for subsequent use by the motion compensation module 904. These 
motion vectors and coding mode decisions 908 are output from the video processing unit 900. 
TTie motion compensation unit 904 generates a compensated frame using reference frames. 
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motion vectors «nd mod. demio.. and subtracts this compensated frame from .he ^compressed 
input frame to yield a difference frame. forward transform unit 910 receives the difference 
frame and performs a fomard spatial transform, such as block-DCT. The quanti^on umt 912 
quantizes .he fransform coefficients produced by fte forwaM transform in ord« to reduce to 
entropy and in doing so may lose some infom^alion. The quanUzed transfom. coefBcenB 914 
are output from the video processing unit 900. The inverse quantization 916 and mverse 
transfbnn 918 uniU repUcate the reconstnH=tion process of a video decoder and produce a 
reference frame that is delivered to the motion compensation unit 904 for optional fumre use. 

Referring next to no. 10, shown is a block diagram of a dual input decoder 1005 
capable of decoding video infom.ation compressed in either format B or fomut A. As shown, 
dual input decoder 1005 includes a firs, enfropy decompression unit 1010 operative to generate 
decompressed intermediate video streams 1012„, which are provided to a switch 1025. Dual 
input decoder 1005 also includes a second entropy decompression unit 1020 configured to 
produce decompressed intermediate streams 1022... which are also provided .o the switch 1025. 
The switeh 1025 selects and relays eiflier intemiediate streams 1012„ from the first 
decompression unit 510 or intermediate streams 1022„ from the second decompression umt 
1020 to the video processing unit 1030 in acconlance with the fonnat being decoded. The vtdeo 
processing unit 1030 then processes the intermediate streams 1012... 1022.. according to well 
known processing techniques so as to generate an uncompressed video stream 1040. 

The dual input decoder 1005 requires fewer resources (e.g. system memory, 
p^gram size, siUcon area, electrical power) than other potential decoding solutions, including, 
for example, a decoder for fonnat A and separate decoder for format B, a decoder for fonnat A 
only and a decompressing converter, and a decoder for format B only and compressmg 
converter. 

Arithmetic Coding 

to some of the embodiments described with reference to FIGS. 1-10. mventive 

arithmetic compression techniques are utilized to effect the format B compression. Tlie 
ariflmtetic codmg techniques acconiing to an exemplary embodiment of the present mventton 
involve flie use of aritiunetic coding to compress two-dimensional bitmaps (l-b,t planes) of 
, compressed content. During tite encoding process, a Context parameter is calculated w«h 
respect to each bit position based upon the neighboring bitmap values surromKiing such posruon. 



15 



to an exemplary embodiment, the Con,e^ parameter may assume values from 0 to 16, 
mclusively, each of which is indicative of a different composition of such neighboring bttmap 

values. . . . u 11 

For example, a Context value of "16» corresponds to the case m which all 

5 neighboring bitmap values are "1", which is usually very unlikely to occur. Each Cc.»«ex< value 
is used as an index into an array of predetermined probability tables utilized in an anthmchc 
encoang process described hereinafter. The resuh of this arithmetic encoding process >s then 
incon^rated within the stream of compressed digital content, which is then transmitted to a 
decoder as an arithmeScaUy compressed stream (e.g., as compressed stream 330, 722, 802) also 
10 referred to herein as a "supercompressed stream." 

At the decoder, the received stream of supercompressed digital content is 
subjected to an arithmetic decompression process. For each bitmap position, the same Cmt^ 
value used during the encoding process is recomputed based upon previously decoded 
nei^boring bitmap values. The recomputed Co.,e., value is used as an index into an array of 
15 predetermined probability tables is id»tieal to the array used during the encoding process. 
The retrieved information is then used to recover flie original compressed digital content (e.g., 
MPEG-4 video) ftom the received stream of supercompressed digital content. 

As shown to Fig. 11, four different eases exist with respect to which the Context 
of bits may be calculated (when scaling from left-to-right and top-bottom). For ttie bits 
20 completely inside the bitinap (shown in FIG. 11(a)), Context can be calculated as: 

Co«tol-l+A + 2»B + 4»C + 8»D 

For me bits on the left edge of the bitinap (shown in FIG. 1 1(b)), Context can be 

25 calculated as: 

Confexf= 1 + 5*A+10*B 

For the bits on the top edge of the bitmap (shown in FIG. 1 1(0), Context can be 

30 calculated as: 

Co/ifexf = l + 10*A + 5*B 
(excluding two left top bits, which will be coded using Context = 0). 

35 
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Finally, for bits on the right edge of the bitmap (shown in FIG. 1 1(d)), Context 
can be calculated as: 

Confex/ = 1+A+10*B + 4*C 

5 

Fntrnp v compression for a 2D arr ay of events 

In an exemplary embodiment, the generic compression scheme described above 
can be applied to two-dimensional video frame information contained in an event matrix. The 
event matrix can have n entries, each of which corresponds to a rectangular block of the video 
10 frame. The blocks are not constrained to be all the same shape or size, and there may be gaps 
between blocks in the array where a decoder knows by other means that no event information is 
expected. 

The statistical characteristics of the event matrix are then analyzed in order to 
facilitate generation of probability table arrays. When encoding an event, the probabiUty table is 
15 selected in accordance with the Context value at the array location corresponding to such event. 
In certain implementations it is possible that more than a single hard-coded probability table 
array may exist for the same data. In such cases, the data is analyzed prior to encoding in order 
to enable appropriate selection of one of the probability table arrays. 

Referring next to FIG. 12, shown is a flowchart illustrating steps carried out 
20 during an exemplary implementation of the inventive arithmetic coding process. The steps set 
forth in FIG. 12 are carried out by a variable length-encoding module (e.g., variable length 
encoding module 444, 446, 448) of an entropy compression unit (e.g., entropy compression unit 
315, 415, 620, 720). The variable length-encoding module (e.g., variable length encoding 
module 444, 446, 448) performs a raster iteration over the all n events in the event matrix, 
25 performing the steps shown in FIG. 12 for each event Ci (i = 1 to n). 

As shown in FIG. 12, at a step 1205 a "special" Context value is generally 
selected and used for the first two elements in the event matrix (which are handled separately, 
since in the exemplary implementation at least two known values are needed to compute 
Context). At a step 1210, the Context value is used as an index into the array of predetermined 
30 probability tables, and the probability table is retrieved. Each entry in the array is a table whose 
entries provide the probabilities of occurrence of all possible values of event Ci. At a step 1215, 
arithmetic coding is performed on the first event using the event's value and the probability table. 
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It is observed that the first and second events are typically processed in the same way as all other 

events, with the exception that the Context values for the these events are set to a predefined 

value used only in connection with these events. 

At a step 1220, a determination is made of whether any fiirther events firom the 

event matrix are to be processed. If so, control passes to a step 1225, in which the next element 
(i.e.. event) is retrieved from the event matrix, hi a step 1230, a Context value is computed from 
a fimction of values of previously processed neighborhood events. At a step 1235, the Context 
value is used as an index into the array of predetermined probability tables. Each entry in the 
array is a table whose entries provide the probabilities of occurrence of all possible values of 
event e^. At a step 1240, arithmetic coding is performed using the probability table and the 
event's value. 

As the events of the matrix are processed in this way, the resulting output from 
the a variable length encoding module (e.g., variable length encoding module 444, 446, 448) is 
incorporated into the compressed data stream (e.g., by the format B multiplexer 450) for the 
present video frame in order to thereby generate a supercompressed data stream (e.g., format B 

compressed stream 330). 

For decoding of the supercompressed stream, the same compressed data stream 
inherent therein is input into an arithmetic coding entropy decompression unit (e.g., the format B 
entropy decompression unit 520). The arithmetic coding entropy decompression iterates over a 
decoded event matrix using the same raster as the variable length encoder (e.g., the variable 
length encoder 444, 446. 448). At each event position, the arithmetic coding entropy 
decompression unit (e.g., the format B entropy decompression unit 520) performs the following 
steps, as detailed in FIG. 13, to produce decoded events e, that are identical to the encoded events 
Ci (for i = 1 to n). 

At a step 1305, a predefined Context value is selected for the first and second 
elements in the event matrix (which, as in the encoding case, are handled separately from other 
events). At a step 1310, a probability is selected for the first element in the event matrix. With 
the Context value and the probability value, arithmetic decoding is performed in a step 1315, 
using a standard arithmetic decoding process. 
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At a step 1320, a determination is made of whether any further events for the 
event matrix are to be reconstructed (i.e., decoded). If so, control passes to a step 1325, in 
which the Context value for the next element (i.e., event) is computed from the existing event 
values in the event matrix being decoded. In a step 1330, the Context value is used as an mdex 
into an array of predetermined probability tables that are identical to the predetermmed 
probability tables used in the encoding process. In a step 1335, the probability value retrieved m 
the preceding step is passed to the arithmetic decompression unit (e.g., the format B entropy 
decompression unit 1320), which uses this information together with the input compressed data 

Stream to compute Ci. 

FIG. 14 shows an example of an event matrix 1400 that can be compressed using 
the inventive arithmetic coding method according to the present invention. In this example, 
event matrix 1400 represents the not_coded event matrix that is an array with dimensions one 
sixteenth of the video resolution. For video with resolution of 64x48 (width x height) then the 
not_coded layer or event matrix could take on the values shown in FIG. 1400. 

The encoder iterates over this matrix as the values would be read (i.e., raster 
iteration or left-to-right and top-to-bottom). Upon reaching the value indicated with [] in FIG. 
14, the Context value can be computed as: 

c[x,y]=l+e[x-ly]+2*e[x-ly-l]+4*e[xy-l]+8*e[x+ly-l] = 9 

In this example, the Context value of 9 will result in statistics table #9 being 
selected Statistics table #9 will most likely show that the probability of finding a 0 is high. The 
statistics information together with the value of interest (i.e., 0) from the event mafrix is passed 
to an arithmetic coding module (e.g., arithmetic coding module 444, 446, 448) within an 
arithmetic coding entropy compression unit (e.g.. entropy compression unit 315, 415, 620, 720) . 

At the decoder, previous binary events in the raster iteration leading up to the 
same example bit above would have already been decoded. Thus, the currently decoded event 
matrix in this example might look like the one shown in FIG. 15, where [?] indicates the value 
that can be decoded next. As in the encoder, a Context value of 9 is computed. Accordingly, the 
exact same statistics table #8 that was used in the encoder can be retrieved. The information now 
available is enough for an arithmetic decompressor to output the value 'O'. 
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In m exemplary embodiment, the compression scheme according to the present 
invention is applied to MPEG-4 p-ftames. which contain «f to 90-95% of all data in a digital 
video stream. Infonnation in a p-ftame c^ be structured into several planes (..e.. "event 
matrices") with different levels of detail. These ev^t matrices, in ISO/IEC 14496-2 specificatton 

5 terminology, are: ^ 

•not coded' - A basic event matrix. It is a two-dimensional bitmap with each bit mdicatmg 
if any further data will be transmitted for a corresponding 16x16 'macroblock'. In MPEG-4 
it is not compressed at all (exactly one bit is transmitted for each entry). 
'mcbpC - This event matrix contains information on several aspects, including: (a) whether 

10 chrominance blocks in this macroblock are coded, (b) the encoding mode of this macroblock 
(eg interorintra),(c)thenumberofmotionvectorsused,and(d)whetherthismacroblock 

is a quantizer change point. For compression purposes the mcbpc event matrix can be split 
into 'intra', 'inter4v', 'cbpc', and 'inter_q' layers. 

'cbpy' - This event matrix contains information on whether luminance blocks of this 

1 5 macroblock are coded. 

'motion_vector' - This event matrix contains information on motion vector or vectors 

associated with the macroblock. 

'acpred', 'dquant', 'mcsel' - This event matrix contains supporting information, and is not 
present in most macroblocks. 
20 'blocks' - This event matrix contains information on quantized DCT coefficients. iWs event 
matrix occupies the most space in P-frames at high bitrates, but is also the least compressible 
one. It can be also split into 'dct' and 'block.sizes' layers, hiformation from 'block_sizes' 
indicates how many codes are present in the certain block, and information from 'dct' tells 
what they actually are. 

25 

tti^rV-mded matrix approach to arithmetic coding 

For each block in a mafrix of blocks, an event e, is a binary value whose meaning 

= 0 indicates an image block at position i is skipped and the block at position i in the 
30 previously decoded video frame is to be output by the decoder instead. 

e, = 1 mdicates motion and/or texture information for this block are included in the 
compressed data stream for this video frame. 



20 



10 



The preferred embodimait for a block-coded matrix compressor includes an 
analyzer tot determines the level of correlation between neighboring blocks in the block-coded 
array. The output of this analyzer is used to select a probability table array that is suited to that 

particular block-coded event matrix. 

The preferred embodiment for a block-coded matrix having blocks of equal size 

and event e., at row column x, uses a raster iterating along each row of the image in sequence. 
Tl,e four closest already-encoded events are used to computeaCon/ex^C, value in the rangeOto 

16: 

Cxy = 1 + ex-iy + 2*ex-iy-i + 4*exy.i + 8*ex+iy-i 

rRPV/r.RPC arith mfttic coding approach 

Unlike in MPEG-4, coded block pattem luminance (CBPY) mformation will be 
stored before coded block pattem chrominance (CBPC) information. CSPY comprises an event 
which has 16 possible values, and which indicates whether luminance texture informatton is 
5 available for a given 8x8 block within a 16x16 macroblock. Similarly. CBPC is an even, which 
contains apptoxhnately 22 possible values. The CBPC event indicates whether chrommance 
torture infomtation exists for a current macroblock, and provides an indication of the type of 
such macroblock (i.e., the manner in which the macroblock is encoded). 

As weU as not coded and intra information, CBPY information can be considered 
,0 a 2. bitmap, to addition, each ftame is divided into a number of -subfiames', each side 
approximately 10-15 macroblocks long. For each subftame. one of 4 statistics tables is selected 
^ditsindexiswritteninu-thebitstream. to one embodiment, at Step 1210 a probability table .s 
selected ftom among sixteen probabUity tables and CBPC is compressed aa=ordmg to the 
selected probability table, which is selected by the value of CBPY of macroblock 

25 

i„Hv..,-nrt«t manix ari'bmrtic coding approach 

For each block in a matrix of blocks, event code e, is a binary value whose 

meaning is: 

ei = 0 indicates image block at position i is coded using inter-frame prediction. 
30 e! = 1 indicates image block at position i is coded using intra-frame means. 
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Intra^ed matrices can be comprised using the general method described 
above, with the following modification. First, the number of intra-coded macroblocks m the 
ftame is detetmined. It is very likely that there will be no mtra^ded macroblocks at aU. or only 
a few Correspondingly, a 2-bit index can be calculated that describes density of mtra-coded 
5 blocks and can take on the following values in an exemplary embodiment of the present 
invention: 

00- indicates no intra-coded macroblocks 

01- ndicates less than 1 intra-coded macroblock per 00 total macroblocks 
10- ndicates less than 1 intra-coded macroblock per 10 tota^ ^^''''^^^'^X. 

10 11: Indicates more than 1 intra-coded macroblock per 10 total macroblocks 

This index is written directly into the bitstream. If there are no intra-coded 
n^acroblocks, no further information needs to be written. Otherwise, arithmetic compression can 
be apphed using one of 3 statistics tables, selected using the index. Macroblocks with not_coded 

15 bit =1 are skipped. . 

Encoding of interAv may be performed using the same method and statistics tables 
as of intra-coded. Macroblocks, which are already known to be intra-coded, cannot be inter4v, so 

they are skipped. ^ 

The preferred embodiment for an intra-coded matrix compressor mcludes an 

20 analyzer that determines the proportion of events in the event matrix, or in a local area of the 
matrix, that have value 1. THe output of this analyzer is used to select a probability table array 
that is suited to that particular intra-coded event matrix. 

The preferred embodiment for an intra-coded matrix having blocks of equal size 
and event e., at row column uses a raster iterating along each row of the image in sequence. 
25 mfourclosestalready-encodedeventsareusedtocomputeaCon^exrc.,valueintherangeOto 

16: 

Cxy = ex-iy + 2*ex.iy-i + 4*exy-i + 8*ex+iy-i 
Blocks known to be skipped can be omitted in the raster scan at both encoder and 

decoder. 

30 Encoding of inter4v may be perfonned using the same method and stausttcs tables 

as of intra, macroblocks. which are already known to be intra (and cannot be inter4v). so they are 

skipped. 
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TUAfinn-rnm pensation ip n'^e matrix arithmftir. coding approach 

For each block i in a matrix of blocks, an event Ci is a vector describing a motion 
vector. The preferred embodiment selects a probability table based on the maximum motion 
vector magnitude for the video frame being coded. The magnitude of the motion vector 
5 component having greater magnitude is entropy coded using the selected probability table. If 
this motion vector component is not zero, its logarithm is used to select a second probability 
table. The magnitude of the remaining motion vector component is encoded using this second 
table. 

The signs of any non-zero motion vector components are signaled in the 
10 compressed data stream using a single bit for each component. If either component is non-zero, 
a bit is written to the data stream to record which motion vector component had the larger 
magnitude. 

The above process may be described in more detail as follows. Specifically, for 
each motion-compensation mode block in a matrix of blocks, event code e* is a binary value 

15 whose meaning is: 

Ci = 0 indicates image block at position i is coded using one motion vector per block. 
Ci = 1 indicates image block at position / is coded using four motion vectors per block. 

Compression of motion vectors may be extended to an /I'ary value where there 
are n different motion compensation modes to be encoded. 
20 hi an embodiment, motion compensation vectors can be calculated by first 

comparing the absolute values of two components of motion vectors. The larger of the two is 
named 'max.code', and smaller of two is named 'min.code'. hi this embodiment, 'max_code' is 
written into the bitstream, using the statistics table, selected with fixed.code frame parameter 
(there are 8 possible values of fixed_code and, correspondingly, 8 different tables), hi this 
25 exemplary embodiment, the fixed_code frame parameter is explicitly sent at the begimiing of the 

bitstream (in both format A and format B). 

If 'max_code' is 0, motion vector is null and no fiirther data needs to be written. 
Otherwise, integer logarithm of 'max_code' (smallest integer N such as 2**i\r >= max_code) is 
calculated. It is used to select one of 8 statistics tables for 'min_code'. 
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After both 'max.code' and 'min_code' are written, up to three bits may need to 
be written: a bit which shows whether x or y component is 'max_code' (if max_code != 
min_code), the sign of x ( if abs(x) != 0 ) and the sign of y ( if abs(y) != 0 ). 

T^vt,ir>.-r.n<1ed matrix arithmetic coding approach 

For each block in a matrix of blocks, an event e^ is a binary value whose meaning 

'\ = 0 indicates no texture is coded for the image block at position / and only motion 

compensation information is used. 
= 1 indicates texttire information for this block is included in the compressed data stream 

for this video frame. 

The preferred embodiment for coding of luminance texture-coded events divides 
the matrix of blocks into regions. The statistics of e. for each region is analyzed in order to select 
a probability table array that is suited to that region. 

The preferred embodiment for coding of chrominance texture-coded events 
generates a Context value based on the values of the collocated luminance texture-coded events. 
This Context value is used to select the optimum entry in a probability table array. 

f^r\^hrv''^''^ -^Hinp ap proach for a nanti/ed bl ock transform coefficients 
20 The preferred embodiment arranges coefficients into a string by ordermg them 

along a predetermined path through the block coefficients (e.g. zig-zag scan). The string is 
truncated at the last non-zero coefficient in the string. The length of the string and its contents 
are encoded separately. 

25 nnantized coeffi ri^'^t string length arithmetic coding approach 

The length of the coefficient string at block i forms an event e^. A Context, a is 
computed based on the number of local blocks with no textture coding and on the distribution of 
the total number per block of non-zero quantized transform coefficients in the video frame bemg 
encoded, c^ forms the index into an array of probability tables. The returned probability table is 

30 passed together with event e. to an arithmetic coding module (e.g., variable length encodmg 
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module 444, 446, 448) within an arithmetic coding entropy compression unit (e.g., entropy 
compression unit 315, 415, 620, 720). 

Q^.nt^-"^ .n^ffirient string v-^^^^ arithmetic coding approach 
5 The coefficient string is converted into a string of events. Each event e^ m the 

string is derived by pairing a non-zero coefficient value with the number of immediately 
preceding consecutive zero coefficient values. For each event e. a Context c. is derived from one 
or more of: 

the position in the coefficient string of the non-zero coefficient in e* ; 
10 the total length of the coefficient string; and 

the absolute level of the previous non-zero coefficient. 

The Context is used as an index into an array of probabiUty tables. The returned 
probability table is passed with e. to an arithmetic coding module (e.g., variable length encoding 
module 444, 446, 448) within an arithmetic coding entropy compression umt (e.g.. entropy 

15 compression unit 315, 415, 620, 720). 

Set forth in detail above are aspects of at least one embodiment of the present 

invention. Each of the features set forth above may be implemented in one system, method, 

and/or computer executable code in accordance with an embodiment of the present invention. 

Alternatively, each of the features set forth above may be separately implemented m different 
20 systems, methods, and/or computer executable codes in accordance with embodiments of the 

present invention. 

Furthermore, the principles, preferred embodimenB, and modes of operation of 
th. present invention have been described in the foregoing description. However, the inventron 
that is intended to be protected is not to be construed as limited to the partrcular embodiments 
25 disclosed Further, the embodiments described herein are to be regarded as iUustratwe rather 
than restrictive. Others may make variations and changes, and equivalents employed, wrmout 
departing from the spirit of fte p,«ent invention Accordingly, it is expressly intended that all 
such variations, changes and equivalents which fall within the spirit and scope of the present 
invention as defined in the foregoing claims be embraced thereby. 
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The previous description of the disclosed embodiments is provided to enable any 
person skilled in the art to make or use the present invention. Various modifications to these 
embodiments will be readily apparent to those skilled in the art, and the generic principles 
defined herein may be applied to other embodiments without departing fi-om the spirit or scope 
of the invention. Thus, the present invention is not intended to be limited to the embodiments 
shown herein but is to be accorded the widest scope consistent with the principles and novel 
features disclosed herein. 
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